Work around documented Linux mmap bug. #152595
Conversation
On Linux, mmap doesn't always zero-fill slack bytes, despite being required to do so by POSIX. If the final page of a file is in the page cache and the bytes past the end of the file get overwritten by some process, those bytes then remain non-zero until the page falls out of the cache or another process overwrites them. Stop trusting that mmap behaves properly on Linux and instead check whether the buffer was indeed properly terminated. If not, fall back to using `read` to read the file contents. This fixes an obscure clang crash bug that can occur if another program (such as an editor) mmap's a source file and writes past the end of the mmap'd region shortly before clang or clangd attempts to parse the file.
|
@llvm/pr-subscribers-llvm-support Author: Richard Smith (zygoloid) ChangesOn Linux, mmap doesn't always zero-fill slack bytes, despite being required to do so by POSIX. If the final page of a file is in the page cache and the bytes past the end of the file get overwritten by some process, those bytes then remain non-zero until the page falls out of the cache or another process overwrites them. Stop trusting that mmap behaves properly on Linux and instead check whether the buffer was indeed properly terminated. If not, fall back to using This fixes an obscure clang crash bug that can occur if another program (such as an editor) mmap's a source file and writes past the end of the mmap'd region shortly before clang or clangd attempts to parse the file. Full diff: https://github.com/llvm/llvm-project/pull/152595.diff 1 Files Affected:
diff --git a/llvm/lib/Support/MemoryBuffer.cpp b/llvm/lib/Support/MemoryBuffer.cpp
index 601f11f6d23c8..ac686b5fb9099 100644
--- a/llvm/lib/Support/MemoryBuffer.cpp
+++ b/llvm/lib/Support/MemoryBuffer.cpp
@@ -501,8 +501,15 @@ getOpenFileImpl(sys::fs::file_t FD, const Twine &Filename, uint64_t FileSize,
std::unique_ptr<MB> Result(
new (NamedBufferAlloc(Filename)) MemoryBufferMMapFile<MB>(
RequiresNullTerminator, FD, MapSize, Offset, EC));
- if (!EC)
- return std::move(Result);
+ if (!EC) {
+#ifdef __linux__
+ // On Linux, mmap may return pages from the page cache that are not
+ // properly filled with trailing zeroes, if some prior user of the page
+ // wrote non-zero bytes. Detect this and don't use mmap in that case.
+ if (!RequiresNullTerminator || *Result->getBufferEnd() == '\0')
+#endif
+ return std::move(Result);
+ }
}
#ifdef __MVS__
|
llvm/lib/Support/MemoryBuffer.cpp
Outdated
| if (!EC) | ||
| return std::move(Result); | ||
| if (!EC) { | ||
| #ifdef __linux__ |
There was a problem hiding this comment.
How sure are we that Linux is the only OS which has this issue? Would it make sense to just remove the ifdef?
There's also a race condition here: if another processes modifies the file while it's mapped, you get undefined behavior. But we can't really do much about that.
There was a problem hiding this comment.
I expect that on any OS, if another process concurrently mmaps the file with MAP_SHARED and writes into the slack bytes, or truncates the file, we will probably crash, but Linux is the only OS that I've found that documents that this can happen even if the file is not concurrently modified. I wouldn't say I'm sure that other OSes don't have similar behavior; maybe they just don't document it?
I'm happy to generalize the check if you think it's worthwhile; as far as I can see, the only real consequence is that this would fault in the final page earlier, which is probably not an especially interesting difference given that we're presumably going to reach it pretty soon anyway.
|
/cherry-pick 85cd3d9 |
|
/pull-request #153486 |
On Linux, mmap doesn't always zero-fill slack bytes (man page), despite being required to do so by POSIX. If the final page of a file is in the page cache and the bytes past the end of the file get overwritten by some process, those bytes then remain non-zero until the page falls out of the cache or another process overwrites them.
Stop trusting that mmap behaves properly on Linux and instead check whether the buffer was indeed properly terminated. If not, fall back to using
readto read the file contents.This fixes an obscure clang crash bug that can occur if another program (such as an editor) mmap's a source file and writes past the end of the mmap'd region shortly before clang or clangd attempts to parse the file.